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Abstract 

In this paper we propose a general spectral theory for tensors. Our proposed factorization 
decomposes a tensor into a product of orthogonal and scaling tensors. At the same time, our 
factorization yields an expansion of a tensor as a summation of outer products of lower order 
tensors. Our proposed factorization shows the relationship between the eigen-objects and the 
generalised characteristic polynomials. Our framework is based on a consistent multilinear 
algebra which explains how to generalise the notion of matrix hermicity, matrix transpose, and 
most importantly the notion of orthogonality. Our proposed factorization for a tensor in terms 
of lower order tensors can be recursively applied so as to naturally induces a spectral hierarchy 
for tensors. 

1 Introduction 

In 1762 Joseph Louis Lagrange formulated what is now known as the eigenvalue - eigenvector prob- 
lem, which turns out to be of significant importance in the understanding several phenomena in 
applied mathematics as well as in optimization theory. The spectral theory for matrices is widely 
used in many scientific and engineering domains. 

In many scientific domains, data are presented in the form of tuples or groups, which naturally 
give rise to tensors. Therefore, the generalization of the eigenvalue-eigenvector problem for tensors 
is a fundamental question with broad potential applications. Many researchers suggested different 
forms of tensor decompositions to generalize the concepts of eigenvalue-eigenvector and Singular 
Value Decomposition. 

In this paper we propose a mathematical framework for high-order tensors algebra based on a 
high-order product operator. This algebra allows us to generalize familiar notions and operations 
from linear algebra including dot product, matrix adjoints, hermicity, permutation matrices, and 
most importantly the notion of orthogonality. Our principal result is to establish a rigorous formu- 
lation of tensor spectral decomposition through the general spectral theorem. We prove the spectral 
theorem for hermitian finite order tensors with norm different from 1. Finally we point out that 
one of the fundamental consequence of the spectral theorem is the existence of a spectral hierarchy 
which determines a given hermitian tensor of finite order. 

There are certain properties that a general spectral theory is expected to satisfy. The most fun- 
damental property one should expect from a general formulation of the spectral theorem for tensors 
is a factorization of a cubic tensor into a certain number of cubic tensors of the same dimensions. 
Our proposed factorization decomposes a Hermitian tensor into a product of orthogonal and scaling 
tensors. Our proposed factorization also extends to handle non-Hermitian tensors. Furthermore our 
proposed factorization offers an expansion of a tensor as a summation of lower order tensors that are 
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obtained through outer products. Our proposed factorization makes an explicit connection between 
the eigen-objects and the reduced set of characteristic polynomials. The proposed framework de- 
scribes the spectral hierarchy associated with a tensor. Finally the framework aims to extend linear 
algebraic problems found in many domains to higher degree algebraic formulations of corresponding 
problems. 

The organization of this paper is as follows; Section [2] reviews the state of the art in tensor 
decomposition and its relation to the proposed formulation. Section [3] introduces our proposed 
tensor algebra for order three tensors. Section [4] introduces and proves our proposed spectral 
theorem for order three tensors. Section [5] discusses some important properties following from the 
proposed spectral decomposition. Section [6] proposes a computational framework for describing 
the characteristic polynomials of a tensor. Section [7] generalizes the introduced concepts to higher 
order tensors and introduces the notion of the spectral hierarchy. Section [8] discusses in details 
the relation between the proposed framework and some existing tensor decomposition frameworks. 
Section [9] concludes the paper with a discussion on the open directions. 



2 State of the art in tensor decomposition 
2.1 Generalizing Concepts from Linear Algebra 

In this section we recall the commonly used notation by the multilinear algebra community where 
a fc-tensor denotes a multi-way array with k indices |17j . Therefore, a vector is a 1-tensor and a 
matrix is a 2-tensor. A 3-tensor A of dimensions m x n X p denotes a rectangular cuboid array of 
numbers. The array consists of m rows, n columns, and p depths with the entry a^.j^ occupying 
the position where the i th row, the j th column, and the fc" 1 depth meet. For many purposes it will 
suffice to write 



A = {a iJjk ) (1 < i < m; 1 < j < n; 1 < k < p) , 

we now introduce generalizations of complex conjugate and inner product operators. 
The order p conjugates of a scalar complex number z are defined by: 



z c p = \Z#t 2 (z) + 3 2 (z) exp <ix arctan 



x exp 



,27Tj 

p 



(1) 



(2) 



where 3 (z) and $ft (z) respectively refer to the imaginary and real part of the complex number z 
equivalently rewritten as 

.2wj 



from which it follows that 



z\ exp ^ i x Z 2 x exp 

= n ^ 

i<i<p 



P 



(3) 



(4) 



The particular inner product operator that we introduce relates the inner product of a p-tuple of 
vectors in to a particular £ p norm operator C' in a way quite similar to the way the inner product of 



pairs of vectors relate to the usual li vector norm. We refer to the norm operator | 
every integer p > 2) as the l v norm defined for an arbitrary vector x = ( x(l), 

by " ' 



Ik : C 



(for 



X L = 



e n w fc )) c? 
i<fe<i i<j<p 



the inner product operator for a p-tuple of vectors in C' denoted 



ii\P 



••• , x(l) )eC l 
(5) 

is defined by 
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kw p = e n Mj)f^ (6) 

l<J<i [o<fc<p-l J 

some of the usual properties of inner products follow from the definition 

((xi + y x ) ; z 2 ; • • • ; z t ) = {xi ; z 2 ; • • • ; z t ) + <y x ; z 2 ; ■ ■ ■ ; zj) (7) 
and most importantly the fact that 

(*;«;•••;*;*)>() (8) 



p operands 

and 

= 0^2 = 0. (9) 



p operands 

We point out that the definitions of inner products is extended naturally to tensors as illustrated 
bellow 

{A,B)= £ a m ,„x(fo„ :m ) c = (10) 

l<m,n<Z 

(A, B 7 C) — y ^ a rn,n,p x (bp.m,n) 3 x ( c ri,p,m) 3 ? (H) 
l<m,n,p<£ 

More generally for arbitrarly finite order tensor the inner product for the family of tensors 
is defined by: 

<n w .- e f n «,,-„.,, w - u )' r ') <»> 

l<ii,i 2 ,-" ,»n<( \0<t<ra-l / 

note that the addition in the indices are performed modulo n. 

Generalization of other concepts arising from linear algebra have been investigated quite extensively 
in the literature. Cayley in [1 instigated investigations on hyperdeterminants as a generalization of 
determinants. Gclfand, Kapranov and Zelevinsky followed up on Cayley's work on the subject of 
hyperdeterminants by relating hyperdeterminants to X-discriminants in their book |10] . 
A recent approach for generalizing the concept of eigenvalue and eigenvector has been proposed by 
Liqun Qi in [30, 28 and followed up on by Lek-Heng Lim|26j, Cartwright and Sturmfels [5 . The 
starting point for their approach will be briefly summarized using the notation introduced in the 
book |10) . Assuming a choice of a coordinate system Xj — (xj(0), Xj(l), ■ ■ ■ , Xj(kj)) associated with 

each one of the vector space Vj = (R + ) We consider a multilinear function / : (S>j =1 Vt —> R + 
expressed by : 

f(xi,X2,---,x r -i,x r )= 22 a iu ... , ir xi{ii) ■ ■ -x r (i r ), (13) 

cquivalently the expression above can be rewritten as 

f(x 1 ,x 2 , ■ ■ ■ ,x r -i,x r ) = (x 1 ,x 2 , ■ ■ ■ ,x r -i,x r ) A . (14) 
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which of course is a natural generalization of bilinear forms associated with a matrix representation 
of a linear map for some choice of coordinate system 



/ (xi,x 2 ) = ^2 a ^<^ xi(h) x 2 (i 2 ) = (xi f Ax 1 = (x 1 ,x 2 ) A . (15) 

It follows from the definition of the multilinear function / that the function induces r not necessarily 
distinct multilinear projective maps denoted by fk : ® r ^ _ i Vt — > Vk expressed as : 

t ^ k 

f k (xi,x 2 , ■ ■ ■ Xk-i,Xk+i, ■ ■ ■ ,x r ) = ^ a lu ... ,i r xi(ii) x 2 (i 2 ) • ■ • x k -i(ik~i)xk+i(ik+i) ■ • ■ x r (i r ) 

it,-" ,ik-l,ik + l--- ,ir 

(16) 

The various formulations of eigenvalue eigenvector problems as proposed and studied in [30, 28, 5, 26 
arise from investigating solutions to equations of the form: 

fk(x,--- ,x) = A • x (17) 

Applying symmetry arguments to the tensor A greatly reduces the number of map fk induced by 
A. For instance if A is supersymmetric (that is A is invariant under any permutation of it's indices) 
then A induces a single map. Furthermore, different constraints on the solution eigenvectors Xk 
distinguishes the ^-eigenvectors from the TJ-eigenvectors and the Z-eigenvectors as introduced and 
discussed in [30, 28_. 

Our treatment considerably differs from the approaches described above in the fact that our aim 
is to find a decomposition for a given tensor A that provides a natural generalization for the concepts 
of Hermitian and orthogonal matrices. Furthermore our approach is not limited to supersymmetric 
tensors. 

In connection with our investigations in the current work, we point out another concepts from 
linear algebra for which the generalization to tensor plays a significant role in complexity theory, 
that is the notion of matrix rank. Indeed one may also find an extensive discussions on the topic of 
tensor rank in |291 IT51 [TBI |3"T1 [B] . The tensor rank problem is perhaps best described by the following 
optimization problem. Given an r-tensor A = {<ii lt ... i r ) we seek to solve the following problem 
which attempts to find an approximation of A as a linear combination of rank one tensors. 



mm 



1<t<r e{.®i<*<r*) 



, Kk<l 



(18) 



Our proposed tensor decomposition into lower order tensors relates to the tensor rank problem 
but differs in the fact that the lower order tensors arising from the spectral decomposition of 3- 
tensors, named eigen-matrices are not necessarily rank 1 matrices. 



2.2 Existing Tensor Decomposition Framework 

Several approaches have been introduced for decomposing fc-tensors for k > 3 in a way inspired by 
matrix SVD. SVD decomposes a matrix A into A = UT,V T and can be viewed as a decomposition 
of the matrix A into a summation of rank-1 matrices that can be written as 

r 

A = J2°i ® (ui,Vi) (19) 

where r is the rank of A, m, Vi are the i-th columns of the orthogonal matrices U and V, and <7j's 
are the diagonal elements of S, i.e., the singular values. Here (&(•, •) denotes the outer product. 
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The Canonical and Parallel factor decomposition (CANECOMP-PARAFAC, also caller the CP 
model), independently introduced by [D E], generalize the SVD by factorizing a tensor into a 
linear combination of rank-1 tensors. That is given A £ I" lX " !X " 3 , the goal is to find matrices 
U e Rmxnx ; V £ R n * xn i and W £ R n ^ xn ^ such that 

r 

A = y"Vj ®(ui,Vi,Wi) (20) 
i=i 

where the expansion is in terms of the outer product of vectors Ui, Vi,Wi are the i-th columns of U, 
V, and W, which yields rank-1 tensors. The rank of A is defined as the minimum r required for 
such an expansion. Here there are no assumption about the orthogonality of the column vectors of 
U, V, and W. The CP decomposition have been show to be useful in several applications where 
such orthogonality is not required. There are no known closed- form solution to determine the rank 
r, or to find a lower rank approximation as given directly by matrix SVD. 



Tucker decomposition, introduced in [34 , generalizes over Eq 20 where an (n\ x n 2 x n%) tensor 
A is decomposed into rank-1 tensor expansion in the form 



Hi "2 1 3 



A = ^2^2^2<7i,j,k ® (ui,Vj,w k ) (21) 
i=l j=l k=l 

where itj £ W 11 , Vj £ R™ 2 , and Wk £ K™ 3 . The coefficients <Ti,j,k form a tensor that is called the core 
tensor C. It can be easily seen that if such core tensor is diagonal, i.e., <7j ,• f. = unless i = j = k, 
Tucker decomposition reduces to the CP decomposition in Eq|20) 

Orthogonality is not assumed in Tucker decomposition. Orthogonality constraints can be added 
by requiring Ui,Vj,Wk to be columns of orthogonal matrices U,V, and W. Such decomposition was 
introduced in [21] and was denoted by High Order Singular Value Decomposition (HOSVD). Tucker 
decomposition can be written using the mode-n tensor-matrix multiplication defined in |21| as 

A = C xr U x 2 V x 3 W (22) 

where x„ is the mode-n tensor-matrix multiplication. Similar to Tucker decomposition, the core 
tensor of HOSVD is a dense tensor. However, such a core tensor satisfies an all-orthogonality property 
between its slices across different dimensions as defined in |21| . 

HOSVD of a tensor can be computed by flattening the tensor into matrices across different 
dimensions and using SVD on each matrix. Truncated version of the expansion yields a lower 
rank approximation of a tensor [22 . Several approaches have been introduced for obtaining lower 
rank approximation by solving a least square problem, e.g. |39| . Recently an extension to Tucker 
decomposition with non-negativity constraint was introduced with many successful applications [32 . 

All the above mentioned decompositions factorizes a high order tensor as a summation of rank-1 
tensors of the same dimension, which is inspired by such an interpretation of matrix SVD as in Eq 
[19] However, none of these decomposition approaches can describe a tensor as a product of tensors 
as would be expected from an SVD generalization. The only known approach to us for decomposing 
a tensor to a product of tensors was introduced in a technical report [16J. This approach is based on 
the idea that a diagonalization of a circulant matrix can be obtained by Discrete Fourier Transform 
(DFT). Given a tensor, it is flattened then a block diagonal matrix is constructed by DFT of the 
circulant matrix formed from the flattened tensor. Matrix SVD is then used on each of the diagonal 
blocks. The inverse process is then used to put back the resulting decompositions into tensors. This 
approach results in a decomposition in the form A = U * S * V T where the product is defined as 

m 

A*B = f old(drc(unf old(A, l)).unf old(B, 1), 1) 

However, such decomposition does not admit a representation of the decomposition into an expansion 
in terms of rank-1 tensors. The product is mainly defined by folding and unfolding the tensor into 
matrices. 
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From the above discussion we can highlight some fundamental limitations of the known tensor 
decomposition frameworks. Existing tensor decomposition frameworks are mainly expansions of a 
tensor as a linear combination of rank-1 tensors, which are the outer products of vectors under 
certain constraints (orthogonality, etc.) and do not provide a factorization into product of tensors 
of the same dimensions. Tucker decomposition, although a generalization of SVD, falls short of 
generalizing the notion of the spectrum for high-order tensors. There is no connection between the 
singular values and the spectrum of the corresponding cubic Hcrmitian tensors. Unfortunately, no 
such relation is proposed by the Tucker factorization. The Tucker decomposition does not suggest 
at all how to generalize such objects as the trace and the determinant of higher order tensors. In 
the appendix of this paper we show that Tucker decomposition and HOSVD uses notion of matrix 
orthogonality. 



2.3 Applications of tensor decomposition 

The most widely used formulation for tensor decomposition is the orthogonal version of Tucker 
decomposition (HOSVD) [21 . HOSVD is a multilinear rank revealing procedure |21[ |2"2"] and there- 
fore, it has been widely used recently in many domains for dimensionality reduction and to estimate 
signal subspaces of tensorial data [IF]. In computer vision, HOSVD has been used in |37l 158] for 
analysis of face images with different sources of variability, e.g. different people, illumination, head 
poses, expressions, etc. It has been also used in texture analysis, compression, motion analysis 
[5S"1 155]. posture estimation, gait biometric analysis, facial expression analysis and synthesis, e.g. 
O [24j I23J [25] , and other useful applications (18) . HOSVD decomposition gives a natural way for 
dealing with images as matrices [39] . The relation between HOSVD and independent component 
analysis ICA was also demonstrated in [7] with applications in communication, image processing, 
and others. Beyond vision and image processing, HOSVD has also been used in data mining, web 
search, e.g. [201 155] . and in DNA microarray analysis |18j . 



3 3-tensor algebra 

We propose a formulation for a general spectral theory for tensors coined with consistent definitions 
from multilinear algebra. At the core of the formulation is our proposed spectral theory for tensors 
. In this section, the theory focuses on 3-tensors algebra. We shall discuss in the subsequent section 
the formulations of our theory for n-tensor where n is positive integer greater or equal to 2. 



3.1 Notation and Product definitions 

A (m x n x p) 3-tensor A denotes a rectangular cuboid array of numbers having m rows, n columns, 
and p depths. The entry (Hj,k occupies the position where the i th row, the j th column, and the k th 
depth meet. For many purposes it will suffice to write 

A := (a^) (1 < i < m; 1 < j < n; 1 < k < p) , (23) 

We use the notation introduced above for matrices and vectors since they will be considered spe- 
cial cases of 3-tensors. Thereby, allowing us to indicate matrices and vectors respectively as ori- 
ented slice and fiber tensors. Therefore, (m X 1 X 1), (lxnX 1), and (1 X 1 X p) tensors indicate 
vectors that are respectively oriented vertically, horizontally and along the depth direction fur- 
thermore they will be respectively denoted by a., 1,1 := (aj,i,i){i< 4 <m}> : = ( a i,j,i){i<j<„}> 
oi,i,. := ( a i,i,fc){i<jfc<j,y Similarly (m X n X 1), (1 X n X p), and (m x 1 x p) tensors indicate that 
the respective martrices of dimensions (m x n), (fixp) and (m x p) can be respectively thought 
of as a vertical, horizontal, or depth slice denoted respectively a.,.,1 := ( a i,i,i)/i<i< m i<j<n}> 
a.,i,. := (ai,i,fe){i<i<m,i<fe< P }' and ai - - : = ( a i j,fc){i<,<„, i< k < P } • 
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Figure 1: Tensor's ternary Product. 



There are other definitions quite analogous to their matrix (2-tensors) counterparts such as the 
definition of addition, Kronecker binary product, and product of a tensor with a scalar, we shall 
skip such definitions here. 

Ternary ■product of tensors: At the center of our proposed formulation is the definition of the ternary 
product operation for 3-tensors. This definition, to the best of our knowledge has been first proposed 
by P. Bhattacharya in [2 as a generalization of matrix multiplication. Let A = (cn,j,k) be a tensor 
of dimensions (m x I x p), B = (&ij,fc) & tensor of dimensions (m x n x I), and C = (ci,j,k) a tensor 
of dimensions (I x n x p); the ternary product of A, B and C results in a tensor D = (dij.k) of 
dimensions (m x n x p) denoted 



D = o(A,B,C) (24) 

and the product is expressed by : 

di,j,k — ^ ai,t,k ■ h,j,t ■ Ct,j,k (25) 
i<t<i 

The specified dimensions of the tensors A, B and C provide constraints for triplet of 3-tensors 
that can be multiplied using the preceding product definition. The dimensions constraints are best 
illustrated by Fig. [2]. There are several ways to generalize matrix product. We chose the previous 
definition because the entries of the resulting tensor D — o (A, B, C) relate to the general inner 



product operator as depicted by Fig.[l]. Therefore, the tensor product in Eq 25 expresses the entries 
of D as inner products of the triplet of horizontal, depth, and vertical vectors of A, B and C 
respectively as can be visualized in Fig. [1]. 

We note that matrix product is a special instance of a tensor product and we shall discuss 
subsequently products of n-tensor where n is positive integer greater or equal to 2. Furthermore 
the proposed definition of the tensor multiplication suggests a generalization of the binary vector 
outer product operator to a ternary operator of slices. The ternary outer product is defined such 
that given tensors A of dimensions (to X 1 X p), B of dimensions (to X n X 1), and C of dimensions 
(lxnx p), their ternary outer product D, noted D — <X>(A, B, C), is an (toX n X p) tensor defined 
by: 



di,j,k — ' ' c l,j,k- (26) 

Note that A, B, and C here are slices arising from oriented matrices. The above definition generalizes 
the binary vector outer product operation to a ternary matrix outer product operation defined by 
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Figure 2: Constraints on the dimensions of the tensors implied by the ternary product definition. 



D = (g) (a.,1,., &.,.,!, ci,.,.) := di^fc = ai,x,fc • &i,j,x • ci Jifc . (27) 

Similarly to matrix multiplication, where the operation of multiplying appropriate sized matrices 
can be viewed as a summation of outer product of vectors, the product of appropriate sized triplet 
of tensors in Eq [25] can be viewed as a summation of ternary outer product of slices 

o(A,fl,C)= ® ("-A- b -,t, ct,.,.). (28) 
i<t<z 

Ternary dot product with a background tensor: The ternary dot product above can be further 
generalized by introducing the notion of a background tensor as follows for ai,.,i = ( a x,i,x){i<i<z}i 

= 0l,l,3') { l<j<J} and = (Cfc,l,l) { i<fe<; } 

(ai,.,i,bi,i,.> c .,M>T := E I! E a i,M' i n, j ' c y,r i .j,t ( 29 ) 
i<i</ \i<j<i \i<fc<; J J 

the preceding will be referred to as the triplet dot product operator with background tensor T. 
Background tensors plays a role analogous to that of the metric tensor. The triplet dot product 
with non trivial background tensor corresponds to a pure trilinear form. Furthermore the outer 
product of 2-tensors can be generalized using the notion of background tensors to produce a 3-tensor 
D which result from a product of three 2-tensors namely a...,i = (a m ,i,x) m v bx,.,. = {°i,n, j) n j and 
c.,x,. = (ck,i,p)k, P as follows, 

d m ,n,v — ^E I ^E I 5E a »",i,l ' ^X,n,j ' c fe,X,p 1 I I • (30) 

l<i<l \l<j<l \X<fc<i J J 

The preceding product expression is the one most commonly used as a basis for tensor algebra in 
the literature as discussed in [5J [M] [7] [TH] . 

We may note that the original definition of the dot product for a triplets of vectors corresponds to 
a setting where the background tensor is the Kronecker delta A = (Sij } k) that is T = A where A 
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Figure 3: Kronecker (2x2x2) tensor. 



denotes hereafter the Kronecker tensor and can be expressed in terms of the Kronecker 2-tensors as 
follows 

Si,j,k = Si.j ■ 5j,k ■ Sk,i (31) 

equivalently A = (<5i,j,fe) can be expressed in terms of the canonical basis {e^ : 1 < i < 1} in l- 
dimensional euclidean space described by: 



A = ^ (e k <g> e k <g> e k ) , 



l<k<l 



hence 



(w, x, y) = (w, x, y) A . 



(32) 



(33) 



3.1.1 Special Tensors and Special Operations 

In general it follows from the algebra described in the previous section for 3-tensors that: 

o (o (A, B, C),D,E)^o (A, o (B, C,D),E)?o (A, B, o (C, D, E)) (34) 

In some sense the preceding illustrates the fact that the product operator is non associative over the 
set of tensors. However tensor product is weakly distributive over tensor addition that is to say 

o([A + B],C, D))=o(A,C,D)+o(B,C,D), (35) 

however in general 

o(A,o(B,C,D),E)+o(A,o(F,G,H),E)^o(A, (o (B, C, D) + o (F, G, H)) ,E). (36) 



Transpose of a tensor: Given a tensor A = (a u ^ v . w ) we define it's transpose A T and it's double 
transpose A T as follows: 



{av,w,u) 



— ^^4. ^ — { a w,u,v) ■ 



(37) 



(38) 



It immediately follows from the definition of the transpose that for any tensor A, A T — A. Inci- 
dentally the transpose operator corresponds to a cyclic permutation of the indices of the entries of 
A. Therefore we can defined a inverse transpose A T = A T , generally we have 
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(39) 



A T " = (A Tq - 1 ) , 
furthermore, a tensor A is said to be symmetrical if : 

A = A T = A T \ (40) 
As a result for a given arbitrary 3-tensor A, the products Z? = o I A, A ,A ) , C = o I A ,A,A 



and D = o ( A 



, A , A J all result in symmetric tensors. It also follows from the definitions of the 
he definition of ternary product operation that: 

(41) 



transpose operation and the definition of ternary product operation that: 

[o{A,B,C)] T = o(B T ,C T ,A T ) 



and 



[o (A,B,C)Y 



B'C\A J 



o(c t \a t \b t2 ). 



(42) 



Adjoint operator: For A G <C mxnxp we introduce the analog of the adjoint operator for 3-tensors in 
two steps. The first step consists in writing all the entries of A in their complex polar form. 



A = {a u , ViW = r u , ViW ■ exp {i ■ u ^ VtW }) (1 < u < to; 1 < v < n; 1 < w < p) . 
The final step expresses the adjoint of the tensor A noted A^ as follows 

T 



(43) 



A f = (-4 C3 ) := (r VtW , u ■ exp {i exp {i ^} • 6 V ^ W . U }) 
A f = (-4 C3 ) == (r w ,u,v ■ exp {i exp {i ^} • 9 w<UiV }) 
= ^A 3 ^ . (a u ,v,w r u,v,w ' exp {i ■ u ,v,w}) 



(44) 



The adjoint operator introduced here allows us to generalize the notion of Hermitian matrices or 
self adjoint matrices to tensors. A tensor is Hermitian if the following identity holds 



A f = A. 



(45) 



Incidentally the products o (A,A t2 ,A f ), o (A^A,^ 2 ) and o (a^,A\a) result in self adjoint 
tensors or Hermitian tensors. 

Identity Tensor: Let l( mxnxp ) denotes the tensor having all it's entries equal to one and of dimen- 
sions (mxnxp). Recalling that A = (<5ij,fc) denotes the Kronecker 3-tensor, we define the identity 
tensors I to be : 



I = (1(1x1x1), l(zxixz), A) = o I l(z x z X i), l(jxfxj), I e k ®e k ®e k 



A<k<l 





<< - 




= s„ iP 






\l<k<l J 





(46) 



(47) 



Furthermore we have : 
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(49) 



5 m , P (51) 



I T = (l(ixixi),A-A(ixixi)) =° l(zxixz), e k®e k ®e k J ,l( ;xix ;) ] (48) 

y yi</c<( 

I T = [I T \ = V] ^m,n.fc = <5m,n 

V ™ \i<fc<< / / 

J T = o (A, l(i x i x i), 1(1x1x1)) = ° E e,£ ® efc ® efc I ' 1 (ix!xO' J-OxJxi) ] (50) 

y yi<fc</ 

^ 2 - f (O - ( E <w.) 

for all positive integer I > 2 . The identity tensor plays a role quite analogous to the role of the 
identity matrix since V A e we have 

o(/,A,/ T2 ) =A (52) 

Proposition 1: VA o ^X, A,X t2 ^ = A and X — (x m , n . p > 0) ^ X = I 

We prove the preceding assertion in two steps, the first step consists of showing that the J is indeed 
a solution to the equation 

VA o(x,A,X T ^=A (53) 

Let R be the result of the product 

R= (r m ,„, p ) =o (/,AJ t2 ) (54) 

^m,n,p — J ^ ^ im,k,p ' &m,n.k ' ^ J J ^ ^ ^/c,p ' ^m,n,/c ' J (55) 

\i<fc<a yi<fe<; J 

r m,n,p = I ^ ' (^fe.p) ' a m,n,k I (56) 



, i<fe<; 



we note that 



I am.n.fe if k=p 
r m ,n,k = < . (57) 
] otherwise 

hence 

A = o(/,A,I T2 ). (58) 

The last step consists in proving by contradiction that 7 is the unique solution with positive entries 
to the equation 



o(x,A,X t2 ^=A (59) 
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Figure 4: Tensor I,I T and I T 



Suppose there were some other solution J with positive entry to the above equation, this would 
imply that 



o(j,a,/ t2 )- o(j,a,j t2 ) 







(60) 



»m,k,p ' a m ,n,fe ' (l T ) I - I y jm,fc,p ' am,n,fc ' ( J T ) (61) 



^ ^ ^m,n,fc 
Kfc<Z 



V / K,n,p 



Jm : k, 



' ( jT2 ) 



(62) 



Since this expression must be true for any choice of the values of a m ,n,k we deduce that it must be 
the case that 



the requirement that 



(o~k,pf - ( jm,k,p ■ (J T ) ) = 

y V / k,n,p J 

=> [jm,k,p ■ \J T ) ) = 5k,p 

\ V / k,n,p J 

jm.k.p — ^&k,p 
jm,k,p 2^ ~^ jm,k,p &k.p 



(63) 

(64) 
(65) 

(66) 



which results in the sought after contradiction □. 

Inverse: By analogy to matrix inverse A^ 1 we recall that for a matrix A, A^ 1 is its inverse if 
(MA) A^ 1 = M, for any non zero matrix M . We introduce here the notion of inverse pairs for 
tensors. The ordered pair (A\, A 2 ) and (Bi,B 2 ) are related by inverse relationship if for any non 
zero 3-tensor M with appropriated dimensions the following identity holds 



M = o (B 1 o (A U M, A 2 ) , B 2 ) . 



(67) 



Permutation tensors: Incidentally one may also discuss the notion of permutation tensors associated 
with any element a of the permutation group S n . 



\/aeS n P cr = o l( nxnxn ),l( n><nxn ), ^ e k ®e k ®e a 



(68) 



, Kk<l 
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= ^2 ° (iCnxnxn), l(nxnxn), (efe ® e fe (g) e (T ( fc ))) (69) 
l<fc<Z 

The 3-tensor P a perform the permutation a on the depth slices of a 3-tensor A through the prod- 
uct o ^P CT , A, P T a ^ . consequently the products o \P^, P T a , Aj and o (^A , P a , \ perform the 
same permutation respectively on the row slices and the column slices of A. 

Proposition 2: Any permutation of the depth slices of A can be obtained by finite sequence 
of product of transposition, and the sequence is of the form 

o (Pa n , • • • , o (p ak , • • • , o (Pa u A, (P^f') • • • , (Pa k f 2 ) , • • • , (Pa n f^ . (70) 

The preceding is easily verified using the definition above and the permutation decomposition the- 
orem [5j. Furthermore permutation tensors suggest a generalization of bi-stochastic matrices to 
bi-stochastic tensors through the Birkhoff-Von Neumann bi-stochastic matrix theorem. 

3.1.2 Orthogonality and scaling tensors 

From linear algebra we know that permutation matrices belong to both the set of bi-stochastic 
matrices and to the set of orthogonal matrices. We described above a approach for defining bi- 
stochastic 3-tensors, we shall address in this section the notion of orthogonality for 3-tensors. We 
recall from linear algebra that a matrix Q is said to be orthogonal if 

Q ] Q = Q Q ] = A. (71) 

When we consider the corresponding equation for 3-tensors two distinct interpretations arise. The 
first interpretation related to orthonormal basis induced by the row or column vectors of the orthog- 
onal matrix Q that is : 

<9.,m>9.,n) = (9m,9n) = ?fc ' m ' q *,n = 5 ™,n ( 72 ) 

The corresponding equation for a 3-tensor Q = (q m ,n,p) of dimensions (I x I x I) is given by: 

A = o(Q,Qt 2 iQ t) (73) 

or explicitly we can write: 

{lm,.,p' Qn,.,m> 1p,.,n) = I Qm,k,p " Qn,k,m ' 1p 3 ,k,n I ~ ^m,n,p- (74) 

\l<k<l J 

The second interpretation arises from the Kronecker invariance equation expressed by: 

A = Q^AQ = (V AQ\ 1 . (75) 
The corresponding Kronecker invariance equation for 3-tensor is given by : 

c i > \ Q,o (q\q^ ,A) ,Q f ) ,Q,Q^ 



(76) 



o(o(Q,o(Qt,Qt ; A),Q t ),Q,Q* 



° (° (Q,° [Q\Q f , A) ,Q f ) ,Q,Q t 
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While Kronecker invariance properly expresses a generalization of the conjugation operation and 
the 3-uniform hypergraph isomorphism equation it does not follow from the first interpretation of 
orthogonality, that is to say 

Q,Q^,Q^^o(o(q,o(q\Q^,a),Q^),Q,Q^=A. (77) 

We now discuss Scaling tensors. The scaling tensor play a role analogous to diagonal matrices in 
the fact that tensor multiplication with scalling tensor results in a tensor whose vectors are scalled. 
First we observe that the identity pairs of tensors should corespond to special scaling tensors. The 
general family of diagonal tensors are expressed by pairs of tensors B = (6 TOl7 j )P ) , C = (c mj „ iP ) such 
that 

B = (6 m ,„, p = 8 n , P ■ Wp^ m ) (78) 

The product D = o (A, B, C) yields 

d m ,n,p = ^ ' a m,k,p ' (&n,k ' w m,k) ' {^k.n ■ w kiP ) (80) 



l<k<l 

~^ d ra n p ^m.n ' Q>m,n.p ' Wn,p (^-0 

The expression above illustrates the fact that w m . n and w n . p scale the entry a m _ n ,p of the tensor 
A, or equivalently one may view the expression above as describing the non-uniform scaling of the 
following vector (a mj „. p ) 1<n<r The vector scaling transform is expressed by 

( a m,n,p)l<„<; {w m ,n 1 a>n,n,p 1 w n,p)i< n <i (82) 

Furthermore the scaling factors for a given vector may be viewed as coming from the same vector 
of the scaling matrix W = (w m n ) if the matrix W is symmetric. Finally we may emphasize the 
analogy with diagonal matrices, which satisfy the following equation independently of the value 
assigned to their non zero entries. For a given D, we solve for C such that 

(D ■ C) m> „ = <„. (83) 

We recall from matrix algebra that: 

C = D (84) 

and furthermore 

D = {d min = <5 m ,„ • w n ) (85) 

(D -D t ) ={ lfm = n (86) 
V / m,n j otherwise 

By analogy we may define scaling tensors to be tensors satisfying the following equation indepen- 
dently of the value of the nonzero tensors. 

(^m,n,p) ^ ^ Q>nx,k,p ' bm,n,k ' c k,n,p (^) 

l<k<l 

a possible solution is given by 
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Q>m,n,p — $m..p ' ^p.n 
^m,n,p $n,p ' ^m,p (^9) 
Cm,n,p — 3m,n ' tVp.rn (90) 

This is easily verified by computing the product 

D = o (A,B,C) = d nh7l]P = ^ ($m,p • w P ,k) • (8 n ,k • Wm,k) • (h,n • w P ,k) (91) 

l<k<l 

^ dni,n,p i$m,p ' ^p.n) ' (^n,n ' ^m,n) ' ($n,n ' ^p,n) (92) 

^ ^m.n.p — {3m,p ' ^p,n) ' ^m,n ' ^p,n (93) 

I otherwise 

Fig[4] provides an example of diagonal tensors. It so happens that A, B, C discussed above are 
related by transpose relation for third order tensors. This fact considerably simplifies the formulation 
of the to diagonality property common to both matrices and 3-tensors. By analogy to matrices we 
say for 3-tensors that a tensor D = (d mi „ ;P ) is diagonal if independently of the value of the non zero 
entries of D we have : 



o(d t ,d t \d) =d 3 m 



Proposition 3: if a 3-tensor D can be expressed in terms of a symmetric matrix W = (w m n = io„ m ) 
in the form D = (d TOi „ iP = w m>n ■ S ntP ) then D is diagonal. 
The proof of the proposition follows from the fact that : 

(D T ) = (w p . n ■ 5 n , m ) (95) 

(D T2 ) = (w n , p ■ 6 p , m ) (96) 

from which it follows that 

o (d t , D t \ D) = {w m . n f ■ 8 n . p (97) 

4 Spectral Analysis of 3-tensors 

Observations from the Eigen- Value/Vector equations. We briefly review well established 
properties of matrices and their spectral decomposition, in order to emphasize how these properties 
carry over to spectral decomposition of tensors. From the definition of eigen- value/ vector equation, 
we know that for a square hermitian matrix A, there must exist pairs of matrices Q, R and pairs 
of diagonal matrices D, E such that 

| A = (DQ? (ER) 

| I = QR y ' 
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Figure 5: Orthogonal slices of an orthogonal tensor 



where the columns of Q' corresponds to the left eigenvectors of A, the rows of -R corresponds to the 
right eigenvectors of A and the entries of the diagonal matrix (^D^'E^j correspond to eigenvalues of 
A. V 

, i 

a m ,n = 2^ (fJ>kQk,m) 2 {Vkr k ,n) ■ (99) 

i<fc<; 

Let f m ,n{k) — <Z fe 3 m • ffc.n; i-e., the entries of the matrix resulting from the outer product of the fc-th 
left eigenvector with the fc-th right eigenvector, incidentally the spectral decomposition yields the 
following expansion which is crucial to the principal component analysis scheme. 

a m ,n = ^2 (^fc 2 ' Vk ) ( 10 °) 
i<fe<i 

The preceding amounts to assert that the spectral decomposition offers for every entry of the 2- 
tensor A a positional encoding in a basis formed by the eigenvalues of the matrix. Assuming that 
the eigenvalues are sorted in decreasing order, the preceding expression suggest an approximation 
scheme for the entries of A and, therefore, an approximation scheme for the 2-tensor A itself. 

Definition The spectrum of an n-tensor corresponds to the collection of lower order tensors the 
entry of which are solutions to the characteristic system of equations. 

Spectrum of Hermitian tensors The aim of this section is to rigorously characterize the spec- 
trum of a symmetric tensor of dimensions (I x I x I). Fig. [5] depicts the product and the slice that 
will subsequently also be referred to as eigen-matrices. 
We may state the spectral theorem as follows 

Theorem 1: (Spectral Theorem for Z-Tensors): For an arbitrary hermitian non zero 3-tensor A 
with ||A||^ 7^ 1 there exist a factorization of the form: 




(101) 



where _D, E, F denote scaling tensors. For convenience we introduce the following notation for 
scaled tensors 



1G 



Q = o(q,D,D t 



R = o[R,E,E T ) (102) 



S = o(s,F,F T ) 
and simply expresses the tensor decomposition of A as: 

A = o (Q,R f ,S^ (103) 
4.1 Proof of the Spectral Theorem 

In what follows the polynomial ideal generated by the set of polynomials {fk}i<k<N 1S n °ted 
)fk(i<k<N- We first emphasize the similarity between the spectral theorem for tensors and ma- 
trices, by providing an alternative proof of a weaker form of the spectral theorem for hermitian 
matrices with Forbenius norm different from 1. Finally we extend the proof technic to 3-tensors and 
subsequently to n-tensors. 

Proof of the weak form of the spectral theorem for matrices 

Our aim is to prove that the spectral decomposition exists for an arbitrary matrix A with forbenius 
norm different 1. For this we consider the ideals induced by the characteristic system of equations 
for matrices. The spectral decomposition of A refers to the decomposition: 

= W^ ER ) , ( 10 4) 
QR y ' 

the spectral decomposition equation above provides us with polynomial system of equations in the 
form 

f a ro ,„ = El<k<l (/* 1k,rnY 2 K r fc ,„) : < m < „ < ; ^ 

conveniently rewritten as 

j ( D ■ q m , E ■ r n ) = a m ,n 
The ideal being considered is : 



1 < m < n < I . (106) 



^ = ) (D ■ q m , E ■ r n ) - a m . n , (q m , r n ) - 5 m . n C C v k ; q k , r k } l < k < l \ ■ (107) 

where the variables are the entries of the pairs of matrices Q, R and 

E) (d m n 5 ra n • /i m ) 

E — (&m,n &m.n ' ^Vn) 

Weak Spectral Theorem (for 2-tensors): For an arbitrary non zero hermitian 2-tensor A with 
|| A\\ e2 7^ 1 the spectral system of polynomial equations : 

j (D-q m ,E-r n ) a 1 < m < n < / (10g) 

admits a solution. 
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Proof : 

We prove this theorem by exhibiting a polynomial p (D, E, q ll r\, 
to the following ideal 



, qi,ri) which does not belong 



2" = ) ( D ■ 9m> E ■ r n) ~ 0. m ,n; (Q m ,r n ) - Sm,n ( 1<m ,n<l ' 



Consider the polynomial 



P (A E, 9l ,n, • • • , q h n) := [ ^ \{D-q m ,E- r n ) 

. l<m,n</ 



-IIAII 



We claim that 



since 



p(D,E, q^n, ■■■ ,q h ri) £1 



p(D,E, qi ,r u - - , qi ,n) Gl^ \\A\\l = \\A\\l 



which contradicts to the assumption that ||^4||^ 2 ^ 1. Hence we conclude that 

\\A\\ 2 e ?l=>p(D,E, 9l ,ri, -- ,q u ri)il 



(109) 

(110) 
(111) 

(112) 



which completes the proof. □ 

In the proof above hermicity played a crucial role in that it ensures that the eigenvalues are not all 
zeros since for non zero hcrmitian 2-tensor A 



Tr {A • A} > 



(113) 



Proof of the Spectral Theorem for 3-tensors 

We procede to derive the existence of spectral decomposition for 3-tensors using the proof thechnic 
discussed above 



A = o (Q,D,D q 



f 2 



o \ R,E,E 

o (q, e) 2 , st) 



(114) 



equivalently written as 



Q>m,n.p — i {f-^m.k ' Qrn,k,p ' l^k.p) ' {^n,k ' ^n.k.m ' ^k,m) 3 ' (£p,fc ' s p,k,n ' £fc,n) 3 / 1 1 _\ 

^ A A ' ' 

>,k,n 



1 r 2 r 1 

c 3 3 

Om,n,p — Z^k=l Qm,k,p ' r n ,k,m ' S p,i 



The variables in the polynomial system of equations are the entries of the 3-tensor Q, _R, S and the 
entries of the scaling tensors D, E, F . 

It is somewhat insightfull to express the system of equations in a similar form to that of matrix 
spectral system of equations using inner product moperators : 



(?m,.,p' ^n,.,m; ^p,.,n) ^m,n,p 

where D UjV is a diagonal matrix whose entries are specified by 



(116) 



(117) 
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The characteristic system of equations yields the ideal I defined by 



) (-^m.p ' Qm,.,pi ^n,m ' ^°n,.,mi ^p,n ' s p,.,ri) a m,n,p] \Qm,.,p> r n,.,mi s p,.,n) °~m,n,p (^< m „ p <; 

: (118) 

where 1 < m, n,p < I. which corresponds to a subset of the polynomial ring over the indicated set 
of variables. The following theorem is equivalent to theorem 1. 

Theorem: (for 3-tensors) If A is a non zero hermitian and ||A||^ ^ 1 then the spectral system of 
equations expressed as 

f (D m , p • q 

I {^m,.,p> ^*n,.,m, ^p,..n) — &7n,n,p 

admits a solution. 



Proo/; 

Similarly to the 2-tensor case, we exhibit a polynomial p which does not belong to the Ideal 1 defined 
bellow. 

) \Dm,p ' Qm..,pi ^n.m ' r n,.,rm Fp,n ' s p,.,n) ~ a m,n,p> {o. m ,.,pi r n,.,mi s p,../n) ~ °~m,n,p \x<m<n<p<l ' 

' : (~ 12 o)" 

Such a polynomial p is expressed by 

»v 

^ I (-^m.p ' Qm,.,pi ^n,m ' ^n,.,m, ^p,n ' &p,.,7l) | J ll-^-ll^ 

v l<i,j,fe<i / 

p£J (121) 

since 

peX=H|A||£ = |l^<, (122) 

which contradicts our assumption that ||A||^ 7^ 1, this completes the proof. □ 

Hcrmiticity also ensure that the solution to the spectral decomposition is not the trivial all zero 

solution since for non zero 3-tensor A 

\\ A \\i = ( E waaa)} w + £ «M, fe -4,r4,i ] >o ( 123 ) 

\l<k<l l<i<j<k<l 

5 Properties following from the spectral decomposition 

Similarly to the formulation for the spectral theorem for matrices, we can also discuss the notion of 
eigen-objects fo r te nsors. I n or der to point out the analogy let us consider the matrix decomposition 
equations in Eq 98 and Eq 99 one is therefore led to consider the matrices Q = (g m! „ = \/\ m q n .m) 
as the scaled matrix of eigenvectors. According to our proposed decomposition, the corresponding 
equations for 3-tensors is given by 

Ec 2 c 1 
(/^m,fc ' Qm,k,p ' ^k,p) ij^n^k ' Tn,k,m ' ^k.m) 3 (£p,fc ' Sp,k,n ' £fc,n) 3 3 (124) 

Kk<l 
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recall that the tensor Q := (q m ,k,p = ^m.fe ■ ^k,p • Qm,k,p) collects as slices what we refer to as the 
scaled eigen-matrices. The analogy with eigenvectors is based on the following outerproduct expan- 
sion. 

A = E (0*0^® (i*r fc ,.)) . (125) 

l<k<l 

The equation emphasizes the fact that a hermitian matrices can be viewed as a sum of exterior 
products of scaled eigenvectors and the scaling factor associated to the rank one matrix resulting 
from the outerproduct corresponds to the eigenvalue. Similarly, a symmetric 3-tensor may also be 
viewed as a sum outer products of slices or matrices and therefore we refer to the corresponding- 
slices as scaled eigen-matrices. The outerproduct sum follows from the identity 

A = o ^Q,R f ,S^ (126) 

expressed as : 

l 

A = ^ ®(q. jfci ., ?.,.,*,§*,.,.), (127) 
fe=i 

which can be equivalently written as 



E ((^m.fc ' IJ>k,p) ( v n,k ■ V k . m ) H (£ p ,fc • 6c,n) C3 ) fm,n,p( k ) ( 128 ) 
l<k<l 

where f m , n , P (k) denote the k-th component expressed 

/m,n,p(^) : — Qm,k,p ij'n.k.m) 3 (Sp,fc,n) 3 ■ (1^) 

We may summarize by simply saying that: as one had eigenvalues and eigenvectors for matrices one 
has eigenvectors and eigen-matrices for 3-tensors. 



6 Computational Framework 

We shall first provide an algorithmic description of the characteristic polynomial of matrix without 
assuming the definition of the determinant of matrices and furthermore show how the description al- 
lows us to define characteristic polynomials for tensors. We recall for a matrix that the characteristic 
system of equations is determined by the algebraic system of equations 

A = Q T DR^\ (r>*-q m ,D*-r n )=a m , n 1 < m < n < ? (13Q) 
as discussed above induces the following polynomial ideal 



1=) (D* . q m ,D2 . r \ amnj (q m ,q n ) - 6, 



C 



Krn<n<l 



(131) 



Let G be the reduced Grobner basis of X using the ordering on the monomials induced by the 
following lexicographic ordering of the variables. 



Q > R > Ai > • • • > A; 



(132) 
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In the case of matrices it has been established that there is a polynomial relationship between the 
eigenvalues; more specifically the eigenvalues are roots to the algebraic equation 

p(A) = dct (A - A • I) (133) 

By the elimination theorem |27| we may computational^ derive the characteristic polynomials as 
follows 

InC[Aj] =det(A-\ l I) (134) 

It therefore follows from this observation that the reduced Grobner basis of I determines the char- 
acteristic polynomial of A. 



Definition 

Let Q denote the reduced Grobner basis of the ideal 2" using the the lexicographic order on the 
monimials induced by the following lexicographic order of the variables. 

Q > R> S > D > E > F 

where 

) (D m p ■ q m m p, E n m ■ V n - m , F n ^ m ■ Sp.n) dm,n,pi { l Qm,.,p1 r n,.,mi s p,.,n) &m,n,p d< m < n <p<; 

The reduced characteristic set of polynomials C associated with the hermitian 3-tensor A is a subset 
of the reduced Groebner basis Q such that 

C :=gr\C[D,E,F] (135) 

where C [D, E, F] denotes the polynomial ring in the entries of the sacaling tensor with complex 
coefficients. The reduced should here be thougth of as generalization of the characteristic polynomial 
associated with matrices. 



7 The General Framework 
7.1 n-tensor Algebra 

An (mi x m 2 x • • • x m n _! x m n ) rt-tensor A is a set of elements of a field indexed by the set 
resulting from the Cartesian product 

{1, 2, • • • , (mi - 1), mi} x {1, 2, • • • , (m 2 - 1), m 2 } x • • • x {1, 2, • • • , (m„ - 1), m„} 

The dimensions of A is specified by (mi x m% x • • ■ x m n _i x m n ) where V 1 < k < n , S N* 
specifics the dimensions of the tensor. We may also introduce a dimension operator defined by 

mu if 1 < k < n , 

d(A '* ) = { o i." (136) 

Finally, we shall simply use the notation convention A = (a il i2 ... ^ ) for describing A once the 
dimensions have been specified. 

In what follows we will discuss general tensor products for rt-tensors where n is a positive integer 
greater or equal to 2. Let us start by recalling the definition of matrix multiplication 

*m.=E«£t°Sa' ( 137 ) 

i 
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the preceding matrix product generalizes to the proposed 3-tensor product as follows 

K,i 2 ,i3 = E a iu3,i3 ' a i%2,3 • a ^l,t3- ( 138 ) 
3 

By closely inspecting the expression of the product we note that if is a (to x k x 1) tensor, and 
is a (fc x n x 1) tensor then the resulting tensor B expressed by 



1 < 2l < TO 

< i2 < n 



(139) 



will be of dimensions (to x n x 1). The product above expresses the action of 3-tensor of 
dimension (to x n x k) on the pair of matrices arising from and A*- 3 -*. Furthermore for A^ 2 ' 
having entries such that 

/ 1 < h < to \ 

A< 2 > = (af 42 . = l) V(ii,i 2 , j) «.t. 1 < * 2 < n , (140) 

V i<j<fc / 

the result of the action of A^ on the pair of matrices arising from the tensorsA'- 1 -' and A^ simply 
corresponds to a matrix multiplication. For 4-tensor the product operator is expressed as : 



b ■ -Va (1) -a {2) -a (3) • a (4) (Ul) 

3 



Similarly the tensor A^ can be chosen to be all-one tensor which reduces the product above to the 
product operation for 3-tensors. This nested relationship will also apply to higher order tensors. 
We may now write the expression for the product of n-tensor. Let = (<4'^ 2i ... j^j j 

denotes a set of n-tensors. The product operator has therefore n operands and is noted: 

B = OIU (AW) (142) 

defined by 

K * 2 - i =Y fa,- 1 * , , x • • • x af\ . . . , x • • • x o[") , ) (143) 



It follows from the definition that the dimensions of the tensors in the set I A^ = (af\ , ] \ must 

I V ll - l2 '-- l "J ) l<t<n 

be chosen so that : 

d(A (1 \ 2) = d(A (2) , 3) = • • • = d^"- 1 *, n) = d(A {n \ 1). (145) 

which describes the constraints on the dimension relating all the n tensors in the product. The 
constraints accross the (n — 1) other dimensions for each tensor are described by the following 
relation. 

d{A {l) ,k) = d{A u \k)Vk(£ {(.7 + 1), (i + 1)} (146) 
The tensor B resulting from the product is a n-tensor of dimensions . 
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(d , l) x d ( A (2) , 2) x • • • x d (a^- 1 ' , (n - 1)) x d ( , n) ) (147) 

TVoie fftorf the product of tensors of lower order all arise as special cases of the general product formula 
describe above. 
Tensor Action: 

The action of n th order tensor A = (a ilii2i ... i j n ) on (n — l)-tuple of order (n — 1) tensors jl?^ = (&i*) 2i 
denned as 

= E ((n^L-A,*,^.--.,*.) 41,.,^) • d48) 

The equation above generalizes the notion of matrices action on a vector. 

Tensor Outerproduct: The outer-product of n-tuple (n — l)-tensors is denoted by : 



B 

I- ! 

and defined such that : 

/ /n-l 



n 

= (g) (AW) (149) 



fciA.-A. = (^(11";':, , J 41,.,^ J • (150) 

The Kronecker n-tensor is defined as 



a = (* 1Iia ,.., i(B _ 1)IiB = ^n<w>j ^,uj =E(^ n ) ( i51 ) 

Order n tensor transpose/adjoint: 

Given a tensor A = (oj 1)-7 - 2i ... the transpose A T is defined such that 

A T = (a j2 , j3 ,... , jndl ) . (152) 
For a complex valued tensor where the entries are expressed in their polar form as follows : 

A = ( a ji,j 2 ,--- ,j n = r h,j2,— jn ' exp{i ■ 9j 1 ,j 2 ,--- ,j n }) j (153) 
the generalized adjoint is given by 

a1 = ( r h,h,-,jn,h ■ expli-expli- ^\ ■ hj3i ... Jn A) , (154) 



n 



A ^ = ( r <r»(ji).^Cj2).-.(j»)" ea: p{ i " eX p{ i " } - ^ ,.-:}) ' (I 55 ) 

where cr^ denotes the composition of k cyclic permutation of the indices from which it follows that 

A r = A. (156) 
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7.2 The Spectrum of n-tensors. 

In order to formulate the spectral theorem for A € C' we will briefly discussed notion of orthogonal 
and scaling n-tensors, which can be expressed as 



0?=r (Q^) (157) 



that is 

/ /n—l 



6 > <••• •• < X! ^1 1 J d, i2 ,-,i J . ( 158 ) 

Where T denotes the transpose operation, which still corresponds to a cyclic permutation of the 
indices. 

We first provide the formula for the scaling tensor whose product with A leaves the tensor 
unchanged. 



(0(A£> (1) ,£> (2) ,£> (3) ,-- - ,£> ( ™- 1) )) (159) 



k 



(160) 

Vt<n-2 ^ t) = (C 2 ,.. )in =W) 

— \ U h,i2,— ,in ~ l1 ' 12 J 

The above family of tensors play the role of identity operator and are related to one another by 
transposition of the indices. The more general expression for the scaling tensors is therefore given 

by 



Vt < n - 2 S (t) = (s^li 2j - ,i n = S i2,i 2+t ■ w lt , l2+t ) 

a(n-l) = — S- ■ ■ U3- ■ \ 



(162) 



where W = (w m . n ) is a symmetric matrix. The expression for the scaled orthogonal tensor is 
therefore expressed by 

o(q,s^,s^,s^,...,s^)) = * iA ,.-a. I II I ( 163 ) 

We therefore obtain that the scaled tensor which will be of the form : 

Q = 0(<Q, S {1 \S {2 \S {3 \- ■ • , (164) 

Theorem 2: (Spectral Theorem for n-Tensors): For any non zero hermitian tensor A € C' such 
that ||A||" 7^ 1, there exist a factorization in the form 



a = 0?=i ( Q] 

_ / + (n+l-*) v 

A = or=i(Ql 

the expression above generalizes Eq|103| 



(165) 
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Proof of the Spectral Theorem for n-tensors 

The spectral decompostion yields the following system of equations 

a = o?=i [Qt J 

a=o^ far 1- *) 



more insightfully rewritten as 



D 



(i) 



^tl,.,*3,— > *n ' ' >*ti»t+2-" , »n ^ »1 ,■ ■ ■ M , ■ ,»t+2 ■ 



... £)(") 

' *2," 



(1) (*) 

9»i,.,»3i"- i *n ' ' ' ' , ^ii,---,h,- -it+2- 



(*) 



where it i t+2 ... i„ is a diagonal matrix whose entries are specified by 



' 1 



D 



(*) 



,»i,2t + 2- 



■,i n ~ {dm,n ' 4+2 "(^) — ^m,n ' <^m,n) 



(n) 



(166) 



(167) 



(168) 



We had already pointed out earlier in the proof for the spectral theorem for 3-tensors that the proof 
technique would apply to n-tensors with norm ^ 1, where n is a positive integer greater or equal to 
2. Similarly we consider the polynomial expression 



P 



E 



D 



(i) 



. a.' ■ ■ ■■■ D yi > - ■ ■ -a {l> ■ ■ - ••• D yi) - ■ a m 

-Hill 

and observe that 

where I defines the ideal iduced by the spectral system of equation since 

pel^\\A\\f n =\\A\\l 

which contradicts our assumption that ||^||" n ^ 1, Hence we conclude that 

\\A\\? n ?l=>ptI 



(t) 



. Kii,- A n <l 



»(*) 



» 



)" 

(169) 
(170) 

(171) 
(172) 



this completes the proof. □ 

The I "slices" of the scaled tensor Q t constitutes what we call the scaled eigen-tensors of A which 
are (n — l)-tensors. 

7.3 Spectral Hierarchy 

We recursively define the spectral hierarchy for a tensor A G C r . The base case for the recursion 
is the case of matrices. The spectrum of an (I x I) matrix is characterized by a set of I scaled 
eigen- vectors. The existence of the spectral hierarchy relies on the observation that the spectrum 
of an order n-tensor A e C' is determined by a collection of /-tuple (n — l)-tensors not necessarily 
distinct. Each one of these /-tuples corresponding to a scaled orthogonal eigen-tensor. By recursively 
computing the spectrum of the resulting scaled orthogonal (n — l)-tensors, one determines a tree 
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structure which completely characterizes the spectral hierarchy associated with the n-tensor A. The 
leaves of the tree will be made of scaled eigenvectors when the spectral decomposition exists for all 
the resulting lower order tensors. 

It therefore follows that the tensor A can be expressed as a nested sequence of sums of outer 
products. We illustrate the general principle with 3-tensors. Let A denotes a third order tensor 
which admits a spectral decomposition in the form described by Eq |127| We recall that the spectral 
decomposition for 3-tensors is expressed by 

A = o (q, R r , 5 f j (173) 

i 

•A = ^ ] & (^{[l"m,k ' Hk,p ' Qm,k,p) m p > i^n.k ' ^k,m ' r n,k,m) n m > {£,p,k ' £,k,n ' r p,k,n) p^ri^j (1^) 
fc=l 

by computing the spectrum of the scaled eigen-matrices we have : 



V 1 < ii < I Q(k) = {li m ,k ■ Hk,p ■ q m ,k :P ) myP = 

i<ji<! 



7ii (k) ■ Uj t (k) 



(175) 



V 1 < h < I = {l>n,k ■ Vk,m ■ r n ,k,m) n , 



Yl ( \l X h • ( k ) ) ® ( \/ X h (k) ■ x h (k) 

\<]2<l 



(176) 



v i < i 3 < i s(k) = (c P , fe • • r P ,k, n ) 7hm = Yl NhM- y 3 A k )) ® ^M) 

l<h<t 

(177) 

where V 1 < k < I , j h (k), {u jl (k), v jl (k)},X h (k),{w h (k), x ]2 (k)} and (3 j3 (k),{y ja (k), z j3 (k)} 
denote the eigenvalues and corresponding eigenvectors respectively for the matrices S(k) ,Q(k), 
R(k). It therefore follows that A can be expressed by the following nested sum of outer product 
expressions 



A = 




8 Relation to previously proposed decompositions 

We shall present in this section a brief overview of the relationship between our framework and 
earlier proposed tensor decompositions 
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8.1 Tucker Decomposition. 

Let us show in this section how the Tucker decomposition in fact uses matrix algebra more specifically 
orthogonality of matrices to express the singular value decomposition for 3-tensors. We used for this 
section the notation and convention we introduced through this work. The Tucker factorization 
scheme finds for an arbitrary 3-tensor D the following decomposition 

D = Tx 1 Q {1) x 2 S {2) x 3 [/ (3) , (179) 

where T denotes a 3-tensor and S^ 2 \U^ 3 ' denote matrices. The product expression used for 
the decomposition written above corresponds to our proposed definition for triplet dot product with 
non trivial background as described in Eq |30| Using our notation we can express the decomposition 
of D as follows: 

d m ,n,p — (flm,i,l, ^„j, Ck,l,p)T = J]] J]] J]] ttm,i,l ' &l,n,j ' c k,l,p 1 U,j,k (180) 

i j k 

Our starting point is the following invariance relation, which arises from the matrix products with 
the identity matrix. 

d m ,n,p = ^ ^ ^ 7m,!,l ' 7l,n,j ' lk,l,p ' di.j^k j (181) 
I j k 

where "/ m ,i.i — Sm.i , 7i,nj = $n,j an d 7M,p = $k,p which correspond to transposes of the identity 
matrix. For any orthogonal matrices Q, S and U we know that 

7l,nj = J2r S ~t,n,r ' SlJ >r (182) 
7fc,l,p = J2v u k,l,v ■ U p ,i iV 

Incidentally the expression in Eq|181|can be written as : 



(183) 



i j k \ y / \ r / \ v / 



by interchanging the order of the sums we get 



^2 I Qm,y,lQi,y,l I ' [ s l,n,r s l,j,r I ' I u fc,l>« ' u p,l,v J ^i,j',fc 

J/ r u \ j / \ 3 J \ k / 



(184) 



we now separate out the products in the expressions to yield the general form of the Tucker decom- 
position. 

=> ^ ^ ^ Qm,y,l ' Sl,n,r ' U Pj l >v I <7i,i/,l ' s lj> 1 «fc,l,i> ' ^ij'.fe I (185) 

y r v \ i j k J 

T = I i;/,r,-u = ^ ^ ^ ' s lj,r ' u /c,l,-u 1 ^ij,fc J (186) 

\ i j k J 

The preceding emphasizes that the Tucker decomposition reuses matrix orthogonality and does 
not provide a generalization of the notion of orthogonality for n-tensors. Finally to determine the 
orthogonal matrices Q, S and U to use we specify the following constraints 
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^ ^ k,g,a ■ U,g,l3 - S a ,P ' E (^,s,a) 2 ] (187) 

is \l,g 

E E ti > a <a ■ *'.i8>s = <w ■ I X! (*'.«.s) ) ( 188 ) 

iff \ i,ff 

E E • = <w • E (*«.',ff) 2 ] ( l89 ) 



which is referred to as the total orthogonality condition. 



8.2 Tensor Rank 1 decomposition. 



The Rank 1 decomposition of tensor |29l IT51 [T51 [3TJ [51 [TTJ [T^] corresponds to solving the following 
optimization problem. Given an r-tensor A = {(H-i,— ,i r ) we seek to find: 



A E ( A *) r 4 t} n (iso) 



mm 1 1 ■' v - »• • ,v ' ""' 



' Kt<r 



Since Johan Hdstad in [?] established the intractability of the tensor rank problem for 3-tensors we 
briefly discuss the relationship to our framework. It follows from the definition of the outer product 
of matrices to form a 3-tensor that 

® (Mi = (m s ,i, t ) s t ,N 1 = (n a , t ,i) a t , Pi = (pi, s ,t) s t ) = -D = (c^j.k = m,-,!,*. • m^i ■ Pi.j,k) ■ 

(191) 

We point out that for the very special matrices essentially made up of the same vector as depicted 
bellow : 

TOi,i, fe = VI < k < I (192) 
n i.j,i = v i.j,i V 1 < i < I (193) 
Pi,j,k = wi,i,k V 1 < j < I (194) 

the outer product of the matrices 

<g) (Mi = (m StU ) s t , Ni = (n s , t ,i) Sit , Pi = (Pi, 8 ,t) g)t ) = « ® t? <E> (195) 

This allows us to formulate the tensor rank problem in Eq |190| in terms of the outer product operator 
for slices as follows 

E ® ( M fc = (A* • ™*,fc,t) S) i , Wfc = (A fc • n.,t,*) Bit , P fc = (A fe • Pfc, S)t ) Sit ) - A (196) 
\i<fc< ; / l3 

<^min||o(M,JV,P) - A\\ e , (197) 
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where M, N, P are 3-tensors arising from the collection of matrices associated with the collection 
of vectors. The preceding naturally related the tensor rank problem to our proposed tensor product. 
Furthermore the generalized framework allows us to formulate the tensor rank problem for n-tensor 
where n is a positive integer greater or equal to 2 as follows 



mm 



(o?=i (mW))-a 



(198) 



One may point out that the spectral decomposition associated with a Hermitian tensor comes quite 
close to the sought after decomposition at the cost of the trading of the requirement that the matrices 
should be rank one to the fact the matrices should arise from scaled eigen-tensors. 



9 Conclusion 

In this paper we introduced a generalization of the spectral theory for n-tensors where n is a positive 
integer greater or equal to 2. We propose a mathematical framework for 3-tensors algebra based 
on a ternary product operator, which generalizes to n-tensors. This algebra allows us to generalize 
notions and operators we are familiar with from Linear algebra including dot product, tensor adjoints, 
tensor hermicity, diagonal tensor, permutation tensors and characteristic polynomials. We proved 
the spectral theorem for tensors having Forbenius norm different from 1. Finally we discussed the 
spectral hierarchy which confirms the intractability of determining the orthogonal vector components 
whose exterior product result in a given n-tensor. 

Starting from the recently proposed product formula in Eq|25]for order 3-tensors proposed by P. 
Bhattacharya in [2] we were able to formulate a general algebra for finite order tensors. The order 
3-tensor product formula suggests a definition for outer product of matrices as discussed in Eq |27| 
it also suggests how to express the action of a tensor on lower order tensors. Most importantly with 
Eq [29] we propose a natural generalization for the dot product operator and a generalization for 
the Riemann metric tensor ideas. Furthermore the tensor algebra that we discuss sketches possible 
approaches to investigate generalizations of inner product space theory. 

One important characteristic of the product operator for tensor of order strictly greater than 
2 is the fact that the product is not associative. Incidentally by analogy to matrix theory where 
the lost of commutativity for matrix product results into a commutator theory and lie Algeras 
which plays an important role in quantum mechanics, the lost of associativity as expressed in Eq |34| 
could potentially give rise to an associator theory or generalizations of lie algebras. Furthermore the 
transpose operator described in Eq[37|emphasizes the importance of the roots of unity in generalizing 
herminian and unitary tensors. The 3-tensor permutation tensors provided a suprising representation 
for the permutation group S n which provide a glimpse at a tensor approach to a representation theory 
as well as a tensor approach to Markov tensor models. 

At the heart of our work lies the concept of orthogonal tensors. We emphasize the fact the 
orthogonal tensors discussed here are generalizations of orthogonal matrices and are significantly 
different from orthogonal matrices. One significant difference lie in the two distinct interpretation 
of the orthogonality property for tensor. The first interpretation expressed by Eq [73] is analogous 
to orthonormal for a set of vectors. The second interpretation relates to the invariance of the 
Kronecker delta tensor under conjugation as expressed in E c]76| Furthermore we have through this 
work provided a natural generalization for the familiar characteristic polynomial using the important 
tool set of Grobner Basis. 

Spectral analysis plays an important role in the theory and investigations of Graphs. Graph 
spectra have proved to be a relatively useful graph invariant for determining Isomorphism class of 
graphs. It seem of interest to note that the symmetries of a graph described by it's corresponding 
automorphism group can also be viewed as depicting a 3-uniform hypergraph which can in turn 
be investigated by through it spectral properties. Determining the relationship between spectral 
properties of a graph and the spectral properties of it corresponding automorphism seems worthy of 
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attention in the context of determining isomorphism classes of graphs. The general framework which 
address the algebra for arbitrarily finite order tensor allowed us to derive the spectral hierarchy. The 
spectral hierarchy induces a bottom up construction for finite order tensor from vectors. This explicit 
construction may in fact prove useful in the context investigations on tensor rank problems which 
also validate as illustrated in Eq |198| our product operator. 

Acknowledgment : 

We are grateful to Emilic Hogan, Professor Doron Zeilberger and Professor Henry Cohn for helpful 
discussion regarding properties of Ideals. The first author was partially supported by the National 
Science Foundation grant NSF-DGE-0549115. 

References 

[1] Cayley Arthur. On the theory of linear transformations. Cambridge Math., J(4):pp.l— 
16, 1845. 

[2] P. Bhattacharya. A new three-dimensional transform using a ternary product. IEEE 
Trans. Signal Processing, 43(12) :pp.3081-3084, 1995. 

[3] Bruno Buchberger. An algorithmic criterion for the solvability of a system of algebraic 
equations. Aequationes Mathematicae 4, pages pp. 374-383, 1970. 

[4] J. Carroll and J. Chang. Analysis of individual differences in multidimensional scaling 
via an n-way generalization of eckart-young decomposition. Psychometrika, 35:283-319, 
1970. 

[5] Dustin Cartwright and Bernd Sturmfels. The number of eigenvalues of a tensor. 
Preprint \afXiv:10044953\ , 2010. 

[6] Lek-Heng Lim and Christopher Hillar. Most tensor problems are np hard. Preprint 
\arXiv:091 1.139^ 2, 2009. 

[7] L. de Lathauwer, B. de Moor, and J. Vandewalle. Independent component analysis 
and (simultaneous) third-order tensor diagonalization. IEEE Transactions on Signal 
Processing, 49:2262-2271, October 2001. 

[8] David S. Dummit and Richard M. Foote. Abstract Algebra. John Wiley & Sons, New 
York, New York, 2003. 

[9] A. Elgammal and C.-S. Lee. Separating style and content on a nonlinear manifold. 
In Proceedings of the IEEE Computer Society Conference on Computer Vision and 
Pattern Recognition (CVPR), volume 1, pages 478-485, 2004. 

[10] I. M. Gelfand, M. M. Kapranov, and A. V. Zelevinsky. Discriminants, Resultants and 
Multidimensional Determinants. Birkhauser, 1994. 

[11] D. Grigoricv. Mutiplicative complexity of a pair of bilinear forms and of the polynomial 
multiplication. Lecture Notes Computer Science vol 64, pages p. 250-256, 1978. 

[12] D. Grigoriev. Multiplicative complexity of a bilinear form over a commutative ring. 
Lecture Notes Computer Science vol 118, pages p. 281-286, 1981. 

[13] D. Grigoriev and A. Razborov Exponential lower bounds for depth 3 arithmetic circuits 
in algebras of functions over finite fields. Appl. Algebra Engrg. Comm. Comput. #6, 
pages 465-487, 2000. 



30 



[14] R. A. Harshman. Foundations of the parafac procedure: Models and conditions for an 
explanatory multi-modal factor analysis. UCLA working papers in phonetics, 1970. 

[15] Johan Hastad. Tensor rank is np-complete. J. Algorithms, ll(4):644-654, 1990. 

[16] M.E. Kilmer, CD. Martin, and L. Perrone. A third-order generalization of the matrix 
svd as a product of third-order tensors. Technical Report Technical Report Number 
TR-2008-4, Tufts University Department of Computer Science, Medford, MA, October 
2008. 

[17] M.E. Kilmer and CD. Moravitz Martin. Decomposing a tensor. SIAM News, 37(9), 
2004. 

[18] Tamara G. Kolda and Brett W. Bader. Tensor decompositions and applications. SIAM 
Review, 51(3), September 2009. In press. 

[19] Tamara G. Kolda, Brett W. Bader, and Joseph P. Kenny. Higher-order web link analy- 
sis using multilinear algebra. In ICDM '05: Proceedings of the Fifth IEEE International 
Conference on Data Mining, pages 242-249, Washington, DC, USA, 2005. IEEE Com- 
puter Society. 

[20] Tamara G. Kolda and Jimeng Sun. Scalable tensor decompositions for multi-aspect 
data mining. In ICDM 2008: Proceedings of the 8th IEEE International Conference on 
Data Mining, pages 363-372, December 2008. 

[21] Lieven De Lathauwer, Bart de Moor, and Joos Vandewalle. A multilinear singular value 
decomposiiton. SIAM Journal On Matrix Analysis and Applications, 21(4):1253-1278, 
2000. 

[22] Lieven De Lathauwer, Bart de Moor, and Joos Vandewalle. On the best rank-1 and 
rank-(rl, r2, rn) approximation of higher-order tensors. SIAM Journal On Matrix 
Analysis and Applications, 21(4):1324-1342, 2000. 

[23] Chan-Su Lee and Ahmed Elgammal. Facial expression analysis using nonlinear de- 
composable generative models. In Proceedings of IEEE Workshop on on Analysis and 
Modeling of Faces and Gestures (AMFG), pages 17-31, 2005. 

[24] Chan-Su Lee and Ahmed Elgammal. Modeling view and posture manifolds for tracking. 
In Proceedings of International Conference on Computer Vision (ICCV), 2007. 

[25] Chan-Su Lee and Ahmed M. Elgammal. Towards scalable view-invariant gait recogni- 
tion: Multilinear analysis for gait. In Proceedings of IEEE Conference on Audio, Video 
Biometric People Authentication (AVBPA), pages 395-405, 2005. 

[26] Lek-Heng Lim. Singular values and eigenvalues of tensors: a variational approach. 
Proceedings of the IEEE International Workshop on Computational Advances in Multi- 
Sensor Adaptive Processing, CAMSAP05(l):pp.l29-132, 2005. 

[27] David A. Cox John B. Little Don O'Shea. Ideals, Varieties, and Algorithms Third 
Edition, 2007. Springer, 2007. 

[28] Liqun Qi. Eigenvalues of a real supersymmetric tensor. Journal of Symbolic Computa- 
tion, 40:pp. 1302-1324, 2005. 

[29] Liqun Qi. Rank and eigenvalues of a supersymmetric tensor, the multivariate homo- 
geneous polynomial and the algebraic hypersurface it defines. Journal of Symbolic 
Computation, 41 (12) :pp. 1309-1327, 2006. 



31 



[30] Liqun Qi. Eigenvalues and invariants of tensors. Journal of Mathematical Analysis and 
Applications, 325:pp. 1363-1377, 2007. 

[31] Ran Raz. Tensor-rank and lower bounds for arithmetic formulas. Proceeding of the 
42nd STOC, 2010. 

[32] Amnon Shashua and Tamir Hazan. Non-negative tensor factorization with applications 
to statistics and computer vision. In ICML '05: Proceedings of the 22nd international 
conference on Machine learning, pages 792-799, New York, NY, USA, 2005. ACM. 

[33] Jian tao Sun, Hua-Jun Zeng, Huan Liu, and Yuchang Lu. Cubesvd: A novel approach 
to personalized web search. In In Proc. of the 14 th International World Wide Web 
Conference (WWW, pages 382-390. Press, 2005. 

[34] L.R. Tucker. Some mathematical notes on three-mode factor analysis. Psychometrika, 
31:279-311, 1966. 

[35] M. A. O. Vasilescu. An algorithm for extracting human motion signatures. In Proc. of 
IEEE C'VPR, Hawai, 2001. 

[36] M. A. O. Vasilescu. Human motion signatures for character animation. In In ACM 
SIGGRAPH 2001, Los Angeles, 2001. 

[37] M. A. O. Vasilescu and D. Terzopoulos. Multilinear analysis of image ensebles: Ten- 
sorfaces. In Proc. of ECCV, Copenhagen, Danmark, pages 447-460, 2002. 

[38] M. Alex O. Vasilescu and Demetri Terzopoulos. Multilinear subspace analysis of image 
ensembles. 2003. 

[39] H.C. Wang and N. Ahuja. Rank-r approximation of tensors: Using image-as-matrix 
representation, pages II: 346-353, 2005. 



32 



